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Abstract 

The impact of Cognitive Coaching SM included as part of the Leader2Leader (L2L) 
Leadership Pilot Program for beginning principals in Alberta, Canada, was evaluated in 
the present study. Fifteen qualified principals (coaches) and 23 new principals completed 
the L2L Pilot Program that took place over 18 months. Questionnaires for coaches and 
new principals were used to obtain reactions to the Cognitive Coaching component. The 
findings suggest that the L2L Leadership Program can help develop a network of reflect¬ 
ive, self-reliant school principals. Recommendations for improving the Cognitive Coach¬ 
ing component of the L2L Leadership Program are provided. 
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Resume 

La presente etude a evalue le volet Cognitive CoachingSM du programme pilote de 
leadership Leader2Leader (L2L) destine aux nouvelles directions d’ecole en Alberta, au 
Canada. Quinze directions d’ecoles d’experience (accompagnateurs/accompagnatrices) et 
23 nouvelles directions d’ecole ont participe au programme L2L Pilot, qui s’est deroule 
sur 18 mois. Des questionnaires pour les accompagnateurs et accompagnatrices et les 
nouvelles directions ont ete utilisees en vue de colliger les reactions au volet Cognitive 
Coaching. Les resultats indiquent que le programme L2L peut aider a mettre sur pied 
un reseau de directions d’ecole autonomes et circonspectes. Des recommandations pour 
ameliorer le volet Cognitive Coaching du programme L2L sont fournies. 

Mots-cles : Cognitive Coaching, evaluation d’un programme, nouvelles directions 
d’ecole, directions d’ecole d’experience (accompagnateurs) 
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Introduction 

Principals, working with teachers, provide the vision for their schools and create a safe 
and nurturing learning environment (Gaziel, 2007; Supovitz, Sirinides, & May, 2010; 
Witziers, Bosker, & Kruger, 2003; Youngs & King, 2002). Importantly, principals foster 
high academic expectations for students (Leithwood & Jantzi, 1999, 2000a, 2000b; Net¬ 
tles & Herrington, 2007). Leithwood, Seashore Louis, Anderson, and Wahlstrom (2004) 
concluded that school leadership “is second only to teaching among school-related factors 
in its impact on student learning” (p. 5). Later, Seashore Louis, Leithwood, Wahlstrom, 
and Anderson (2010) indicated that there have been no cases of improvement in the level 
of student achievement without effective school leadership (see also Kaplan, Owings, & 
Nunnery, 2005; Leithwood & Jantzi, 2000a; Marzano, Waters, & McNulty, 2005; Sebas¬ 
tian & Allensworth, 2012; Supovitz, et ah, 2010). 

In order to better prepare new principals to meet these expectations, their own 
learning needs to be properly addressed through professional development (Pedder, 

2006). Individual and collaborative work should be combined to achieve this goal. The 
purpose of the present study was to conduct a program evaluation of the process of Cog¬ 
nitive Coaching SM (Costa & Garmston, 1994, 2002) included as part of the professional 
development Leader2Leader (L2L) Leadership Pilot Program for beginning principals in 
the province of Alberta, Canada. 


Professional Development 

Development of school leadership occurs as problems emerge and leaders acquire the 
ability to transfer knowledge from known solutions to new situations. Professional 
development designed to foster this expertise is most effective if it meets both immediate 
and long-term needs of school leaders. Time is needed to absorb, practice, discuss, and 
adapt knowledge to their working contexts (Garet, Porter, Andrew, & Desimone, 2001; 
Guskey, 2000). Effective professional development addresses the personal nature of 
learning, thus accounting for the individual needs of professionals (Clarke & Hollings¬ 
worth, 2002). 

Spanneut, Tobin, and Ayers (2012) studied the professional development needs 
of 273 elementary, middle, and high school principals in the State of New York. Using a 
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four-point Likert-type scale, participants rated their professional development needs with 
respect to 31 functions identified in the six Interstate School Leaders Licensure Consor¬ 
tium (ISLLC) Standards. They also rated the effectiveness of eight professional develop¬ 
ment delivery methods. Spanneut et al. reported that the principals strongly indicated that 
they wanted to identify their own professional development needs with respect to the 31 
functions, a finding that led Spanneut et al. to conclude that principals should have “au¬ 
tonomy” in selecting professional development that aligns with their own plans. The most 
preferred delivery methods included workshops, mentoring/coaching, small study groups, 
and conferences. The least-preferred delivery methods were university coursework on¬ 
line, university course work on campus, and self-paced online learning. 

A mentoring/coaching program operated by the Ontario Ministry of Education 
was based on a theoretical framework consistent with cognitive theory and adaptive ex¬ 
pertise (Nanavati & Robinson, 2009). Both one-on-one coaching and group experiences 
positively influenced new principals’ capacity in the areas identified in the Leadership 
Framework for Ontario. Nanavati and Robinson (2009) concluded that new principals 
improved their skills and their sense of confidence as new administrators by having the 
opportunity to network and meet other administrators in training sessions and group 
meetings to overcome the culture of isolation that often accompanies the first year of 
administration. 


Cognitive Coaching SM 

Initially developed to support teachers, Cognitive Coaching (Costa & Garmston, 1994, 
2002, 2012) involves a non-judgmental and confidential relationship between a coach 
and teacher based on authenticity, honesty, respect, and empathy. 1 Cognitive Coaching 
does not intend to change overt behaviour through counselling, providing advice, or by 
telling teachers what to do. Instead, Cognitive Coaching focuses entirely on developing 
internal thought processes and self-directedness of the teacher being coached. The coach 
ensures that empowering the teacher to be self-sufficient is the focus of each coaching 
conversation. 


1 Cognitive Coaching™ is a service-marked term, but for literary purposes, the service mark will not appear through¬ 
out the remainder of this document. 
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Coaching maps guide the three types of conversations that comprise the Cognitive 
Coaching process: 

a. Planning conversations help teachers being coached to clarify goals for a 
lesson, identify approaches and strategies for instruction, and create an as¬ 
sessment plan about how they will assess their students. They are encouraged 
to establish a personal learning focus and begin to articulate processes for 
self-assessment. They are then given a chance to reflect on the planning pro¬ 
cess and to explore and refine their ideas. 

b. Reflection conversations provide opportunities for teachers to reflect on their 
lessons and how the implementation of the teaching and assessments worked. 
They are encouraged to identify causal factors and create new meaning. They 
are also given a chance to reflect on the impact of the reflecting conversation 
and to explore and refine their thoughts. 

c. Problem-solving conversations begin with the existing state of the teacher. 

The coach first acknowledges the existing state, frames the desired state with 
the teacher, and then assists the teacher to draw on his or her own resources or 
locate new resources to reach the desired state. 

Coaches create a safe and non-judgmental environment in which they carefully 
listen, follow the agenda of the teacher, paraphrase what the teacher says, allow silence 
and space for reflection, and offer feedback in the form of a probing question or a pos¬ 
sible solution presented in the form of a question. The intent is to develop self-directed 
teachers who are self-managing, self-monitoring, and self-modifying (Costa & Garmston, 
2012 ). 

Showers (1984, 1985) found that teachers who were coached demonstrated the 
newly learned strategies and skills more effectively than teachers who were not coached. 
Joyce (1987) found that whereas only 5% of the teachers learned a new skill and incorpo¬ 
rated it into their instructional practice when only theory was taught, added demonstration 
increased the implementation level to 10%, added practice increased the implementation 
level to 20%, feedback during practice increased the implementation level to 25%, and 
Cognitive Coaching used in conjunction with theory, demonstration, practice, and feed¬ 
back increased the implementation level to 90%. Slinger (2004) reported that “Cognitive 
Coaching was responsible for developing deeper and stronger relationships with peers as 
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well as with their students” (p. 57). Coy (2004) found “self-directed learning was evident 
for mentors and proteges in that they recognized their strengths and weaknesses and ei¬ 
ther sought solutions to their dilemmas or modified their behaviours” (p. 148). Similar to 
the findings of Joyce (1987), Batt (2010) reported that of the teachers who participated in 
a training program for a new program, 53% implemented the program following training 
alone and 100% implemented the program when Cognitive Coaching was included as 
part of their training. Loeschen (2012) found clarification in the thinking and practice of 
teachers as mentors supported teachers and that teachers developed their own capacity to 
self-reflect, leading to modified beliefs and actions. 

Although Cognitive Coaching initially started with teachers, Cognitive Coaching 
has been successfully used with school principals (Ellison, 2003; Ellison & Hayes, 2005). 
Qualified coaches were experienced principals who had completed the Cognitive Coach¬ 
ing Foundation Program. The Foundation Program consists of 40 hours of training, which 
includes working individually with one principal in need of support. Following training, 
the Cognitive Coaching takes place over one or two years, depending on the availabil¬ 
ity of time for both the coach and the principal being coached. The process is basically 
the same as the process used with teachers, with each coach working with the principal 
being coached. The intent is to develop self-directed principals who are self-managing, 
self-monitoring, and self-modifying. 

Olivero, Bane, and Kopehnan (1997) included workshops and Cognitive Coach¬ 
ing with 31 managers and supervisors employed in a health agency in the State of New 
York. The participants attended a three-day workshop on management that included a 
variety of interactive activities and focused on their work roles. The workshops were 
followed by eight weeks of one-on-one Cognitive Coaching. The managers and supervi¬ 
sors met with their coach once a week for one hour. The topics discussed included goal 
setting, collaborative problem solving, practice, feedback, supervisory involvement, 
and evaluation of end-results. Olivero et al. reported that while the three-day workshop 
increased the productivity of the managers and supervisors by 22%, the addition of the 
eight-week one-to-one Cognitive Coaching increased their productivity by 88%. 
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Evaluation Questions 

The program evaluation was focussed on the Cognitive Coaching that took place during 
the L2L Leadership Pilot Program in Alberta during the school years 2012-2013 and 
2013-2014. The following two questions were addressed: 

1. What are the impressions of the coaches and new principals about Cognitive 
Coaching included in the L2L Leadership Pilot Program? 

2. What changes took place in the schools with new principals during the L2L 
Leadership Pilot Program? 

Workshops, which were included as part of the pilot program and were developed in 
response to a needs assessment of the coaches and new principals, were individually 
evaluated at the end of the workshop as a regular part of the Alberta Teachers’ Associa¬ 
tion’s professional development programs. These evaluations are not part of this program 
evaluation. 


Method 


Pilot L2L Leadership Program 

Given the importance of school leadership and the projected retirement of a number of 
current principals, the Alberta Teachers’ Association initiated a 16-month pilot of the L2L 
Leadership Program to support newly appointed principals. The purpose of the L2L Lead¬ 
ership Program was “to develop a network of reflective, self-reliant school leaders whose 
high-quality leadership optimizes student learning and supports improvement initiatives 
that take into account the unique context of each school” (Alberta Teachers’ Association, 

2012, p. 1). 

The L2L Leadership Pilot Program, which was based on the design successfully 
used by Olivero et al. (1997), consisted of 

a. four 1- or 2-day professional development workshops held in January 2013, 
May 2013, September 2013, and January 2014, 
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b. Cognitive Coaching sessions, four of which were held during the second to 
fourth workshops. Given the distances between the coaches and their new 
principals, face-to-face, telephone, email and Skype conversations were used 
between workshops, 

c. a 2-day Leadership Essentials for Administrators Conference held in Novem¬ 
ber 2013 and intended for school administrators in their first or second year, 
and 

d. a celebration day during which the new principals presented either the results 
of their personal growth plan or a plan for changes in their schools. 

In addition, two refresher workshops for the coaches were held—one before the 
May 2013 professional development workshop and the other at the time of the confer¬ 
ence held in November 2013—to address issues raised by the coaches (e.g., conduct of a 
problem-solving conversation). Coaches were asked to use Cognitive Coaching in their 
interactions with each new principal and to avoid evaluating, providing consulting to, and 
collaborating with the new principals they were coaching. 

Evaluation Design 

While the best evaluation design to address the effectiveness of Cognitive Coaching 
would have been to have two random samples of new principals (experimental design) or 
two similar samples of new principals (quasi-experimental design) to allow examination 
of cause-and-effect relationships, the start-up time and costs of the pilot L2L Leadership 
Program precluded use of either of these designs. Consequently, the evaluation design 
involved the coaches as the control group and new principals as the treatment group. 2 

Participants 

Fifteen experienced principals in Alberta volunteered to serve as coaches. Of the 15 
principals, 13 had completed the Cognitive Coaching program (40 hours over eight days) 
and two were currently completing the Cognitive Coaching program. Two of the 29 new 


2 While it would also have been beneficial to collect data from teachers, students, and parent/guardians, the Alberta 
Teachers’Association’s ethics policy precluded collecting these data. 
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principals who applied to participate in the pilot program did not receive approval from 
their district superintendent. Further, before the beginning of the 2013-2014 school year, 
five new principals left the pilot program (no longer a principal [1], wanted consulting 
[1], personal or family reasons [3]). 

During the first introductory workshop held in December 2012, the coaches and 
new principals intermingled in a series or professional development activities, after which 
the new principals provided a list of coaches they would like to work with. The list was 
used to form coaching pairs subject to the condition that a coach and new principal could 
not be from the same or adjoining school district so as to protect privacy. Eight coaches 
individually coached two new principals, and seven coaches coached one new principal. 

Questionnaires 

A questionnaire for the coaches and a questionnaire for the new principals were con¬ 
structed to obtain infonnation about their schools, their impressions of Cognitive Coach¬ 
ing, and themselves. The intent of the questions included in the questionnaires was to 
detennine 

a. how well the Cognitive Coaching was being conducted, 

b. what the coaches and new principals saw as strengths of Cognitive Coaching, 

c. what recommended changes the coaches and new principals had to improve 
Cognitive Coaching of new principals, and 

d. what changes were in the schools of the new principals from the beginning of 
the L2L Leadership Pilot Program and the time at which the questionnaires 
were administered. 

While the two questionnaires contained the same three parts (description of the 
school, impression of Cognitive Coaching, description of self), the questions were not 
always the same due to the differences in experience with and knowledge of Cognitive 
Coaching between the coaches and new principals: 

1. The questions designed to obtain a description of the school were the same in 
both questionnaires. 
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2. Some questions about impressions of Cognitive Coaching were the same for 
the coaches and new principals (e.g., incidence of evaluation, consulting, col¬ 
laboration, and/or coaching). 

Other questions were similar (e.g., will coaches continue if the L2L 
Leadership Program is offered after the pilot and will new principals take the 
training to become a coach). 

The remaining questions were tailored for coaches and new princi¬ 
pals (e.g., while the coaches were asked how many planning, reflection, and 
problem-solving conversations were held and how confident they were in 
conducting the activities associated with each conversation, the new principals 
were asked if specific behaviours associated with planning, reflection, and 
problem-solving conversations occurred). 

3. Of the 11 questions about self, seven were common for coaches and new 
principals (e.g., changes in behaviour—principals know what and how they 
are thinking about their work and are aware of the consequences of their ac¬ 
tions). The remaining four questions about self were for new principals (e.g., 
compared to when they started as a principal, how well they felt they were 
prepared and how confident they were to address instructional issues, manage 
school operations and resources, attend to personnel issues, administer school 
discipline, and interact with parents/guardians and the community at-large). 3 

Since the new principals did not know the process of Cognitive Coaching at the 
beginning of the pilot program, the questionnaires were sent once and returned by email 
at the end of January 2014. The response rate was 100%. A follow-up telephone interview 
was conducted with each coach and new principal during the first three weeks of Febru¬ 
ary. The purpose of the interview was to clarify responses to different questions for each 
principal being interviewed (e.g., omitted items, clarification of written responses). 

The coaches and new principals were asked in the questionnaires to make up to 
three recommendations for improving the L2L Leadership Program based on their par¬ 
ticipation in the pilot program. To allow all the coaches and new principals to review the 


3 Greater detail about the questionnaires is provided with the results. Copies of the questionnaires and the tables with 
the full set of results can be obtained from the first author. 
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21 recommendations made and to either endorse or not endorse each recommendation, 
the recommendations were shared with all of the coaches and new principals by email. 
During this review, one new recommendation was made by a coach. This recommenda¬ 
tion was shared with the coaches with a request to either endorse or not endorse it. The 
response rates were 100%. 

Analysis 

The program directors and members of the Advisory Committee for the L2L Leadership 
program wanted to identify what changes might be needed before full implementation 
of the program. Therefore, univariate rather than multivariate inferential tests were used 
given the seriousness of the consequence of a Type II error compared with a Type I error. 
The independent /-test was used to compare the coaches and new principals for common 
questions. The dependent /-test was used to compare changes between the beginning of 
the L2L Leadership Pilot Program and the time at which the questionnaires were adminis¬ 
tered. The 0.05 level of significance was used and effect size A (Cohen, 1988) was com¬ 
puted for each significant difference. The analyses were completed using SPSS 23. 


Results and Discussion 

Degrees and Years of Experience as a Principal 

Of the 15 coaches, 13 completed a Bachelor of Education (BEd) degree, two completed 
combined degrees such as a BEd with a Bachelor of Arts (BA), two completed a Bache¬ 
lor of Physical Education (BPE) degree, and one completed a Bachelor of Science (BSc) 
degree. Sixteen new principals completed the BEd degree, four completed a combined 
degree, and one each completed a BPE degree, BSc degree, or a Bachelor of Management 
degree. The total number of majors was 24, with the most frequent being social studies (n 
= 7 principals), elementary generalist (n = 5), and physical education (n = 5). 

Thirteen coaches and 12 new principals completed master’s degrees. Of the 13 
coaches, 11 had a Master of Education (MEd), one had a Master of Arts (MA), and one 
had a Master of Science (MSc). Of the 12 new principals, 10 had an MEd, one had an 
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MA, and one had a Masters of Business. The areas of specialization of these coaches 
and new principals included curriculum and instruction, educational leadership, fami¬ 
ly psychology, finance, professional development, and technology integration, with the 
most popular being educational leadership (eight coaches and seven new principals). 

One coach had completed an EdD degree, and one new principal was enrolled in a PhD 
program. 

Whereas 15 coaches had from two to 17 years of experience as a principal prior 
to the 2012-2013 school year, 11 new principals indicated that they were in their first 
year as a principal, nine indicated they were in their second year as a principal, and three 
indicated they were in their third year as a principal. 

Of the 15 coaches, five were principals in schools in rural locations, four prin¬ 
cipals were in small urban schools located in cities with less than 100,000 people, one 
principal was in a rural/urban school (rural-like district next to a city), and five principals 
were in cities with 100,000 or more people. The corresponding distribution for the new 
principals was 13 new principals in schools in rural locations, eight new principals in in 
small urban schools located in cities with less than 100,000 people, one new principal 
was in a rural/urban school (rural-like district next to a city), and one new principal in a 
city. 

Impressions of Cognitive Coaching 

Form and frequency of contact. A new aspect of the pilot L2L Leadership Pro¬ 
gram was the use of other technologies in conjunction with face-to-face contact between 
the coaches and new principals. The coaches and new principals were encouraged to 
communicate using Skype or a similar program, telephone, email, and/or texting, depend¬ 
ing on the geographical distance between them. 

Unfortunately, the number of times the different forms of contact reported by the 
coaches and new principals were not in close agreement. Despite the discrepancies, the 
most frequent form of contact was by email followed by telephone and face-to-face meet¬ 
ings. Skype and texting were used infrequently. 

However, in agreement with the findings of Spanneut et al. (2012), it is clear that 
both the coaches and new principals preferred personal contact. More than 90% of both 
the coaches and new principals endorsed the recommendation to increase the number 
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of face-to-face contacts. But if the recommendation to begin Cognitive Coaching with a 
series of consecutive weekly meetings of coach and new principal at the beginning of the 
program (e.g., meet once a week for two or three consecutive weeks at the beginning) is 
accepted, then the use of Skype may be more palatable. 

Type of interaction. The coaches and new principals were asked how frequently 
their interaction involved a coach evaluating the new principal (evaluation), providing 
advice in response to a request from a new principal (consulting), working with the 
new principal much like co-equals (collaboration), and empowering the new principal 
to become self-directed, self-monitoring, and self-modifying (Cognitive Coaching). As 
shown in Table 1, all coaches and new principals indicated that general interaction never 
involved evaluation. Three coaches and 10 new principals perceived their interaction to 
be consulting, at least most of the time. A nearly equal number of coaches perceived that 
they were engaged in collaboration a few times (10) and at least most of the time (11); 
in contrast, whereas six new principals perceived they were engaged in collaboration a 
few times, 16 perceived they were engaged in collaboration most of the time or always. 
Lastly, while the majority of coaches indicated they used Cognitive Coaching in the 
interactions with their new principals most of the time, 14 new principals indicated they 
were involved in Cognitive Coaching always. Given that the total of the frequencies for 
the new principals is greater than the total frequency of coaches, it is clear from the inter¬ 
views that the coaches moved between collaboration and coaching during a conversation. 


Table 1 . Type of interaction 



Never 

A Few Times 

Most of the 
Time 

Always 

Evaluation 





Coach 

23 




New Prin. 

23 




Consulting 





Coach 

1 

19 

2 

1 

New Prin. 

1 

12 

5 

5 

Collaboration 





Coach 

2 

10 

8 

3 

New Prin. 

1 

6 

8 

8 
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Never 

A Few Times 

Most of the 
Time 

Always 

Cognitive Coaching 





Coach 


5 

15 

3 

New Prin. 


1 

8 

14 


Whether a coach should consult or a new principal wanted consulting was raised 
in the written recommendations and the telephone interviews. Further, as mentioned 
earlier, one new principal dropped out prior to September 2013 because she wanted con¬ 
sulting. Given new principals are just starting out, it may well be that they are not always 
looking for Cognitive Coaching but rather for immediate answers. Further, 100% of both 
the coaches and new principals endorsed the recommendation to clarify the nature of 
consulting and Cognitive Coaching and to allow consulting when it was most appropri¬ 
ate. Coaches should feel without prejudice that they have the flexibility to consult when 
appropriate and to engage in Cognitive Coaching when appropriate, and the new princi¬ 
pals should have knowledge of this expectation. 

Balance of interaction. The coaches and new principals were asked to indicate 
the degree to which a coach or a new principal directed his or her interactions using a 
7-point scale (ratings ranged from 1—Directed by me; 7—Directed by the new principal 
for the coaches; 1—Directed by coach; and 7—Directed by me for the new principals). 
The /-test for independent groups revealed there was no statistical difference between the 
responses of the coaches and the new principals (t = -0.93). 

• Eleven coaches reported their new principals directed more of the interactions; 
four indicated they and their new principals equally directed the interactions; 
and seven indicated that they directed more of the interactions. 

• Fifteen new principals indicated that they directed more of the interactions; 
six new principals said the interactions were balanced; and two felt the coach 
directed more of the interactions. 

These findings reflect somewhat the frequencies of the general nature of the interactions. 
A new principal would receive information and advice through consultation in contrast 
to Cognitive Coaching, which would be at least balanced if not directed more by a new 
principal than by a coach. 
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Number and issues discussed during planning, reflection, and problem-solving 
conversation. The coaches reported that there were 73 planning conversations, 86 reflec¬ 
tion conversations, and 78 problem-solving conversations. 

The coaches and new principals were asked to identify the issues they discussed 
in their conversations. By far the greatest number of issues were related to providing 
instructional leadership 4 (n = 44). Of the instructional leadership issues, dealing with 
staff issues was most common (17) followed by the nature of instructional leadership (8), 
supervising and evaluating teachers (6), and professional growth plans (6). Managing 
school operations and resources and embodying visionary relationships had the next larg¬ 
est numbers of issues, 17 and 16, respectively. The two most common managing school 
operations and resources issues were related to personnel—new principal in a new school 
(3) and dealing with a teacher who was on leave of absence (3). The two most common 
embodying visionary leadership issues were developing the school vision and mission 
(7), and planning and developing professional development (4). Issues and concerns 
related to developing and facilitating leadership and fostering effective relationships were 
mentioned 12 and 10 times, respectively. The most frequent developing and facilitating 
leadership issue was working with the school’s parent council, particularly the chair of 
the council (8). The most frequent fostering effective relationships was working with 
agencies like Child and Family Services (6). Five issues related to leading a learning 
community, and two issues related to understanding and responding to the larger societal 
context were identified, each with a frequency of one. 

The finding that the greatest number of issues identified by far was related to 
providing instructional leadership is likely because this competency is more related to 
student learning and progress than the other six practice competencies. While principals 
need to keep current about provincial and school board policies and regulations, their 
main focus should be to ensure the development and maintenance of effective educa¬ 
tional programs and teaching within their school so as to enhance student learning and 
achievement. 


4 The seven areas are organized in tenns of the seven leadership dimensions included in the Alberta Professional 
Practice Competencies for School Leaders ( PPCSL\ Alberta Education, 2013). 
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Nature of the conversations. The coaches and new principals were asked to 
indicate the frequency of 24 different behaviours associated with Cognitive Coaching 
conversations (Costa & Garmston, 2002, pp. 60-61, 69, 125) that occurred during their 
conversations (1—Did not use; ..., 5—Used very frequently). Items were stated in terms 
of the coach in the questionnaire for coaches (what a coach indicated he or she used) and 
in terms of the new principal in the questionnaire for new principals (what the new prin¬ 
cipals indicated what the coach used). 

Inspection of the means in Table 2 reveals that, for the most part, the various be¬ 
haviours associated with Cognitive Coaching were used frequently. With two exceptions, 
which were both for the coaches and new principals (behaviours xxi and xxii), the means 
rounded to a whole number were 4 or 5. In the case of behaviour xxi —As a coach did you 
ask your new principal how his/her thinking now compared to when you began coach¬ 
ing ?—10 coaches indicated they did so infrequently. For xxii —Did you ask your new 
principal about specific things about your conversation that were helpful? —one coach 
indicated not used, four indicated used infrequently, and 13 indicated about half the time. 
Both of these behaviours are reflective in nature and coaches may have thought they were 
inappropriate to use more frequently during the 10-month coaching period. 

Significant differences between the coaches and new principals were found for 
the first five behaviors and last three behaviors in Table 2 (p < 0.05). For all eight items, 
the mean frequency of use was greater for the new principals than for the coaches and the 
effect sizes were moderate (between 0.50 and 0.80) for six of the items and large (greater 
than 0.80) for two items (items ii and xxiv). There is no clear reason for these differences. 
Further, despite these differences, with two exceptions, it is clear that the behaviours as¬ 
sociated with the process of Cognitive Coaching were frequently to very frequently used. 

Table 2. Nature of conversations 


As a coach, did you... /As a coachee, did your 
coach... 


n 

Mean 

SD 

t 

i. Provide you with adequate time for you to 

Coach 

23 

4.17 

ESS 

2.14* 

state the goal(s) you wished to work on or 
the problems(s) you wished to solve? 

New Prin. 

23 

4.57 

1 

0.62 a 

ii. Pause for an adequate amount of time to 

Coach 

23 

■■ 


3.60* 

allow you to formulate your thoughts? 

New Prin. 

23 

US 

ESI 

0.94 

iii. Provide you with adequate uninterrupted 

Coach 

23 

4.13 

EH 

2.78* 

time to express yourself? 

New Prin. 

23 

4.45 

ESI 

0.46 
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As a coach, did you... /As a coachee, did your 
coach... 


n 

Mean 

SD 

t 

iv. 

Lead you by asking you to consider the 

Coach 

23 

3.57 

1.38 

2.14* 


potential value of different points of view 

New Prin. 

23 

4.30 


0.53 


regarding an issue you had? 






V. 

Ask you about what guided you when mak¬ 

Coach 

23 

3.79 

ESI 

2.09* 


ing a decision about an issue you had? 

New Prin. 

23 

4.35 


0.59 

vi. 

Acknowledge and clarify what you said or 

Coach 

23 

4.52 

EE! 

0.71 


provided? 

New Prin. 

23 

4.65 



vii. 

Summarize and organize what you said or 

Coach 

23 

4.35 


1.07 


provided? 

New Prin. 

23 

4.57 



viii. 

Shift the level of abstraction during a con¬ 

Coach 

23 

3.48 


1.07 


versation with you? 

New Prin. 

22 

3.82 



ix. 

Invite you to envision potential solutions? 

Coach 

23 

4.22 

m 

0.53 



New Prin. 

23 

4.35 

H 1 


X. 

Invite your New Prin(s). to envision who 

Coach 

23 

3.65 


1.32 


might be involved in working out solu¬ 

New Prin. 

23 

4.04 




tions? 






xi. 

Ask you the sequence of steps you might 

Coach 

23 

3.91 


0.58 


take for solutions that required more than 

New Prin. 

23 

4.09 




one step? 






xii. 

Ask you what you previously found ef¬ 

Coach 

23 

3.57 

IE1 

1.31 


fective when you faced situations like the 

New Prin. 

23 

4.00 




present situation? 






xiii. 

Ask you what might be some of the strate¬ 

Coach 

23 

4.09 


0.69 


gies you considered when seeking a pro¬ 

New Prin. 

23 

4.26 




posed solution? 






xiv. 

Ask you what criteria might be used to 

Coach 

23 

3.48 


1.17 


judge your proposed solution? 

New Prin. 

23 

3.87 



XV. 

Ask you what you hoped to accomplish 

Coach 

23 

4.13 

wm 

1.47 


with your proposed solution? 

New Prin. 

23 

4.48 



xvi. 

Intentionally explore your thinking? 

Coach 

23 

4.00 

MM 

1.40 



New Prin. 

23 

4.35 



xvii. 

Intentionally ask you to specify your think¬ 

Coach 

23 

3.74 


1.74 


ing? 

New Prin. 

23 

4.22 



xviii. 

.Set aside unproductive patterns of listening, 

Coach 

23 

4.09 


-0.80 


responding, and/or inquiring? 

New Prin. 

22 

3.82 



xix. 

Navigate between different types of conver¬ 

Coach 

23 

3.74 


1.68 


sations? 

New Prin. 

22 

4.18 
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As a coach, did you... /As a coachee, did your 
coach... 


n 

Mean 

SD 

t 

xx. Ask you if the conversation supported your 

Coach 

23 

3.57 

na 

1.05 

thinking? 

New Prin. 

23 

3.96 

HI 


xxi. Ask how your thinking now compared to 

Coach 

23 

3.00 

na 

1.41 

when you began the coaching? 

New Prin. 

23 

3.52 



xxii. Ask you about specific things about your 

Coach 

23 

3.13 

HI 

2.41* 

conversation that were helpful? 

New Prin. 

23 

3.96 


0.78 

xxiii. Feel your coachee(s) listened carefully to 

Coach 

23 

4.48 

in 

2.48* 

what you said to them? 

New Prin. 

23 

4.91 


0.59 

xxiv. Feel your coachee(s) let you guide the con¬ 

Coach 

23 

3.96 

Esa 

2.95* 

versations you had? 

New Prin. 

23 

4.65 

EH 

0.84 


> < 0.05 


Paraphrasing. An important part of the process of Cognitive Coaching involves 
a coach paraphrasing an issue or concern mentioned by the new principal. The intent is 
to allow the new principal to hear what was said in a different way and then allow time 
to ponder, with the hope that he or she will see the issue or concern in different ways and 
come up with one or more solutions on his or her own. In effect, it is a part of the new 
principal becoming self-regulated. 

The coaches were asked how often they engaged in paraphrasing during a plan¬ 
ning, reflection, and problem-solving conversations (1—Rarely; 2—About half the time; 

3—Always) and how confident they were in doing so (1—Very unconfident; ..., 5—Very 
confident). The coaches reported always using paraphrasing with 15 of the 23 new prin¬ 
cipals during planning and reflection conversations. Paraphrasing for the remaining eight 
new principals happened about half the time. For problem solving, paraphrasing was 
used always with 13 new principals and about half the time with five new principals. The 
coaches were confident to very confident using paraphrasing with all but two new princi¬ 
pals during planning and problem-solving conversations and all but five new principals 
during a reflection conversation. 

The new principals were asked how often (1—Not used; 2—Used infrequently; 

3—Used a lot) their coach used different paraphrases obtained from the Cognitive Coach¬ 
ing learning guide (Costa & Gannston, 2012, p. 55). The paraphrases the new principals 
indicated were used are listed in Table 3 in decreasing order in terms of the frequency 
for “3—Used a lot.” As shown, 21 of the 23 new principals reported the coaches used 
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the paraphrase “So it is important to you that...” and 18 reported the coaches used the 
paraphrase “So you are thinking that ...” most frequently. Seven of the 11 paraphrases 
were used by the coaches a lot (13 or more) with at least half of the new principals, with 
the remaining used less frequently. 


Table 3. Frequency of the coaches’ use of different paraphrases 



1 

2 

3 

So it is important to you that... 

1 

1 

21 

So you are thinking that... 

2 

3 

18 

What are you hoping to accomplish with 
? 

1 

6 

16 

What may be some of your options? 

1 

6 

16 

How might you know when you have reached your 


9 

14 

goal? 




What might be some approaches or strategies that you 

3 

6 

14 

have used before that were effective? 




A goal for you is... 

3 

7 

13 

So an example of what you are talking about is... 

3 

9 

11 

Given the outcome, how do you think the teachers felt 

6 

7 

9 

about what was done and the outcome? 




An assumption you are operating from is... 

7 

11 

5 

First you are going to..., and then move onto... 

9 

8 

5 


Involvement of new principals in conversations. The new principals were asked 
about their involvement (1—No involvement; 2—I think so; and 3—Yes involvement) in 
seven specific activities associated with planning, reflecting, and problem solving conver¬ 
sations. As shown in Table 4, they were 

• well involved in clarifying goals or problems to be addressed, followed by 
anticipating approaches, strategies, and decisions the new principals would 
need to make to address their goals or problems (means = 2.91 and 2.74, 
respectively), 

• less sure about their involvement in establishing a personal learning focus 
and the process of self-assessment as they worked to achieve their goals or 
solve their problems (2.52), reflecting on the coaching process (2.48), and in 
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identifying success indicators that would tell them if they had achieved their 
goals or solved their problems (2.43), and 
• more uncertain about their involvement in developing a plan to collect evi¬ 
dence for a success indicator (2.00) and using the evidence to see how well 
they did (1.96). 

The last results agree with the exceptions noted above for the coaches. 


Table 4. Activities new principals were involved 


Activity 

Responses 

Mean 

SD 

No 

Think so 

Yes 

Clarified your goal(s) or the problem (s) 
you wanted to address 


2 

21 

2.91 

0.29 

Established a personal learning focus 
and a process of self-assessment as you 
worked to achieve the goal(s) or solve 
the problem(s) 


11 

12 

2.52 

0.51 

Anticipated approaches, strategies, and 
decisions you would need to make in 
addressing the goal(s) or solving the 
problem(s) you had 


6 

17 

2.74 

0.45 

Identified success indicators that would 
tell you if you achieved the goal(s) or 
solved the problem(s) you had 

2 

9 

12 

2.43 

0.66 

Developed a plan for collecting the evi¬ 
dence for each success indicator 

5 

13 

5 

2.00 

0.67 

Used the evidence collected to see how 
well you did 

7 

10 

6 

1.96 

0.77 

Reflected on the coaching process 

3 

6 

14 

2.48 

0.73 


Confidence of coaches conducting conversations. The coaches were asked how 
confident they were conducting planning, reflection, and problem-solving conversations 
(1—Not at all confident; ..., 5—Very confident) at the beginning of the pilot program and 
at the end of January 2014. Whereas the coaches were uncertain regarding their confi¬ 
dence at the beginning of the project, they were confident by the end of January 2014. 
This increase in confidence between the two times was significant at the 0.05 level of 
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significance for all three conversations. Further, the interviews revealed that the confi¬ 
dence of the coaches increased rapidly from the beginning to the point they were confi¬ 
dent by the May 2013 professional development workshop. 

The coaches were asked how confident they were at the end of January 2014 in 
engaging each of the elements of planning, reflection, and problem-solving conversations. 
The coaches were confident or very confident for five of the six behaviours associated 
with a 


• planning conversation (clarify goals; specify success indictors and plan data 
collection; anticipate approaches, strategies and decisions and how to monitor 
them; identify personal learning focus and processes for self-assessment; and 
reflect on the coaching process and explore refinements); 

• reflection conversation (summarize impressions and recall supporting infor¬ 
mation/data; analyze causal factors; construct new learning; commit to appli¬ 
cation; and reflect on the coaching process and explore refinements). 

And they were confident or very for confident for three or four of the five behaviours 
associated with a 

• problem-solving conversation (clarify an existing state; establish desired state; 
and identify and amplify resources to achieve desired state) at the end of Janu¬ 
ary 2014. 

The one exception for each conversation involved exploring refinements, which required 
evidence to be collected. 

Rapport and trust. Both the coaches and new principals rated the level of rapport 
and trust between them using a 7-point semantic differential. The anchors were tense and 
rapport for the rapport, and no trust and complete trust for the trust. The means for coach¬ 
es and new principals, which were, respectively, 6.45 and 6.52 for rapport and 6.43 and 
6.57 for trust, were not significantly different. Clearly, there was a high level of rapport 
and trust between the coaches and the new principals. 

Level of trust with others. The coaches and new principals were asked what their 
perceptions were of the level of trust between themselves, their teachers, their students, 
the parents/guardians of the students, and their office staff. A 5-point scale (1—No trust; 
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..5—Complete trust) was used. The t-test for independent groups revealed that there 
were no differences between the means of the coaches and new principals at the 0.05 
level of significance. The means, which varied between 4 and 5, indicated a high degree 
of trust between the principals, be they a coach or a new principal, and their teachers, 
students and their parents/guardians, and office staff. 


Impact of Cognitive Coaching on School Leadership 

Changes in New Principals and Their Schools 

The new principals were asked about the changes in their schools and themselves and 
others since the beginning of the L2L Leadership Pilot Program. A three-point scale (1— 
No, went down; 2—No, stayed the same; 3—Yes, went up) was used. 

• Provincial test scores went up in 10 schools, stayed essentially the same in 
eight schools, and went down in three schools. 

• Both their own self-efficacy and the school’s collective efficacy increased in 
21 schools and stayed the same in two schools. 

• While 19 new principals felt they were more reflective and were thinking in 
more complex ways, four new principals felt that they were now less reflective 
and thought in less complex ways. 

• Sixteen new principals were more satisfied at choosing to become a principal, 
five indicated there had been no change in satisfaction, and two indicated that 
they were less satisfied. 

• School climate and collaboration among teachers increased in 13 and 16 
schools, respectively, and stayed the same in seven and five schools. 

• Of the 23 new principals, 22 indicated that Cognitive Coaching benefited 
them professionally and 18 indicated Cognitive Coaching benefited them 
personally. 

• New principals in 16 schools indicated that the Cognitive Coaching that they 
had received benefited their teachers, their students, and the students’ parents, 
respectively, and another eight indicated there had been no change in benefits. 
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The new principals were also asked to compare how well prepared (1—Very 
much less prepared; ..4—Very well prepared) and how confident (1—Very much less 
confident; ..., 4—Very confident) they now felt to address instructional issues, manage 
school operations and school resources, attend to personnel issues, administer student 
discipline, and interact with the parent/guardians and the community at-large. The val¬ 
ues of the means for preparedness ranged from 3.04 (manage school operations) to 3.39 
(administer student discipline), and for confidence from 3.27 (attend to personnel issues) 
to 3.64 (administer student discipline), indicating that as a group, the new principals felt 
they were now better prepared and more confident for each of the seven competencies. 

Level of Knowledge 

The coaches and new principals were asked to rate how often (1—Not at all, ..., 5—All 
the time) they knew nine activities principals should be involved in. The t-test for inde¬ 
pendent groups revealed that there were no significant differences between the means of 
the coaches and new principals. Further, the vast majority of coaches and new principals 
indicated they often knew: 

• what and how they are thinking about their work and the consequences of 
their actions (14 coaches, 22 new principals), 

• they and their staff would benefit from working together to improve student 
learning and achievement (15 coaches, 22 new principals), 

• they and their staff were willing to change and create relationships that would 
benefit their work (15 coaches, 20 new principals), 

• they, as principals, had the capacity to and the responsibility for initiating 
changes in their schools to improve student learning (15 coaches, 21 new 
principals), and 

• they, as principals, would continue to improve, had options to consider, and 
could acknowledge the opinions from teachers and others regarding instruc¬ 
tion in and operation of their schools (15 coaches, 22 new principals). 

The finding that there were no differences between the coaches, who were experienced 
principals, and the new principals, who were in second, third, or fourth year as a principal 
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at the time the questionnaire was completed, speaks to the effectiveness of the interac¬ 
tions between the coaches and the new principals with which they worked. 


Summary and Recommendations 

Taken together, the findings of the program evaluation of the Cognitive Coaching compo¬ 
nent of the L2L Leadership Pilot Program reveal that the coaches were able to use Cog¬ 
nitive Coaching properly and clearly and that new principals were able to receive Cog¬ 
nitive Coaching positively and to their advantage. In agreement with Ellison (2003) and 
Ellison and Hayes (2005), the findings support the use of Cognitive Coaching to change 
the behaviour of new principals so that they become better prepared as a principal. The 
knowledge, practice, level of thinking, self-reflection, self-efficacy, and confidence of the 
new principals improved during the time of the pilot program. 5 

In contrast to the studies in which incremental changes in actual behaviour were 
assessed with teachers (Batt, 2010) and managers and supervisors (Olivero et ah, 1997), it 
was not possible to do the same in the present program evaluation. However, when asked 
which of the workshops or coaching was of greater value using a 7-point scale anchored 
at each end (1—Workshops greater; ..., 7—Coaching greater), 11 new principals indi¬ 
cated that workshops and coaching were of equal value, and 11 new principals favoured 
coaching. Additionally, all 15 coaches and 23 new principals would recommend the L2L 
Leadership Program to a new beginning principal. Taken together, these findings indicate 
that the L2L Leadership Program should continue with both workshops and Cognitive 
Coaching but not without consideration of the recommendations they made and endorsed. 
Of the 21 recommendations, the following 10 related to Cognitive Coaching were en¬ 
dorsed by at least three-quarters of both the coaches and new principals: 

• Provide new principals with a primer on the Cognitive Coaching process prior 
to the beginning of the coaching/mentoring process so that new principals are 
better informed about the process at the beginning of the coaching/mentoring 


5 Some of the coach-new principal pairs continued to meet following the end of the L2L Pilot Program, which could 
help to improve the confidence of the new principals, thereby increasing their self-efficacy and competency as 
principals. 
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process (e. g., one-day workshop with an introduction followed by modelling 
in which the workshop leader works with one of the new principals, lunch 
question-and-answer session, second modelling but with a different new 
principal). 

• Begin coaching with a series of consecutive weekly meetings of the coach and 
new principal (e.g., meet once a week for two or three consecutive weeks fol¬ 
lowed by a one- or two-week break and then once of week for a further two or 
three consecutive weeks) to create a safe, trusting, honest, and non-judgmental 
environment. 

• Establish guiding expectations or structure for coaches and new principals 
to follow (e.g., establish guidelines for establishing a schedule for contacts 
(weekly, biweekly, monthly; face-to-face, telephone, email); need for Cogni¬ 
tive Coaching and consulting; procedures to follow when the coach or new 
principal cannot make a scheduled contact. 

• Increase the number of face-to-face meetings. 

• Pair coach and new principals from adjoining school districts or within a 
region so as to facilitate face-to-face meetings. 

• Encourage visits between schools of the coach and new principal, so that the 
coach and new principal are aware of and have greater understanding of the 
context in which each works. 

• Provide financial support for coach-new principal pair travel so as to facilitate 
face-to-face meetings and allow school visits where the travel distance is quite 
long. 

• Summarize each coach/new principal interaction at the end of each session 
with questions such as the following: 

i. How well did the session go? 

ii. What, if anything, needs to be discussed further at our next session? 

iii. Did any new issues come to mind as a result of today’s session? 

• Hold a debriefing session at the end of the coaching process. 

• Review the length of fonnal time for the coaching (e.g., is one year sufficient 
assuming the process starts at the beginning of a school year or are two or 
more years needed?). 
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